Proto-tai: Quick Design Prototyping Using Tangible Assisted Interfaces

In the real world, we use our innate manual dexterity to create and manipulate 3D objects. Conventional virtual design tools largely neglect this skill by imposing non-intuitive 2D control mechanisms for interacting with 3D design models. Their usage is thus cumbersome, time consuming and requires training. We propose a novel design paradigm that combines users’ manual dexterity with the physical affordances of non-instrumented and ordinary objects to support virtual 3D design constructions. We demonstrate this paradigm through Proto-TAI, a quick prototyping application where 2D shapes are assembled into 3D representations of ideated design concepts. Here, users can create 2D shapes in a pen-based sketch medium and use expressive handheld movements of a planar proxy to configure the shapes in 3D space. The proxy provides a metaphorical means for possessing and controlling the shapes. Here, a depth sensor and computer vision algorithms track the proxy’s spatial movement. The 3D design prototype constructed in our system can be fabricated using a laser cutter and physically assembled on-the-fly. Our system has vast implications in many design and assembly contexts, and we demonstrate its usability and efficacy through user studies